Multi-processor flash memory storage device and management system

ABSTRACT

A data storage device has a host controller interface, a plurality of microprocessor units each having a portion of random access memory (RAM) dedicated thereto, a plurality of Flash device configurations each having dedicated bus connections to individual ones or multiples of the microprocessor units, and a dataflow controller accessible to the host controller interface for managing access to the Flash device configurations.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present invention is a continuation-in-part (CIP) to a U.S. patentapplication Ser. No. 11/439,619, filed May 23, 2006 and entitled “HybridSolid State Disk Drive with Controller”, to a U.S. patent applicationSer. No. 11/439,620, filed May 23, 2006 and entitled “Software Programfor Managing and Protecting Data Written to a Hybrid Solid State DiskDrive”, and to a U.S. patent application Ser. No. 11/439,615, filed onMay 23, 2006 and entitled “Methods for Managing Data Writes and Reads toa Hybrid Solid State Disk Drive”; disclosures of which are incorporatedin their entireties at least by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is in the field of data storage devices includingdisk drives and mass storage systems and pertains particularly toprocessor-based data storage devices and systems for managing hostaccess to and data management on those devices.

2. Discussion of the State of the Art

In the field of data storage, non-volatile mechanical disk drives havebeen developed for short and long-term data storage. Solid-statenon-volatile memory has been implemented for specific data storageneeds, especially in small, portable electronic computing devices suchas cellular telephones, video cameras and the like.

Volatile memory is a solid-state memory typically used as a high-speedtemporary memory such as random access memory (RAM) of which there aremany known variations. Common versions of RAM include Dynamic RandomAccess Memory (DRAM) and Static Random Access Memory (SRAM) among othervariations such as SDRAM.

Flash memory, on the other hand, is a solid-state, high-speed datastorage solution that has, until recently, been used mainly for handhelddevices like cell phones, personal digital assistants (PDAs), cameras,or Universal Serial Bus (USB) peripheral storage devices referred to asjump drives or thumb drives. Flash memory provides a non-volatile memoryfor storing data with read speeds approaching that of RAM. Common memorytypes that require management include Phase Change Memory and NANDFlash.

When referring to these memory types, the terms volatile andnon-volatile are blurring as new research in memory continues and newmemory types are developed. But for purpose of this specification,volatile memory shall refer to memory in which stored data is lost uponinterruption of power and non-volatile memory shall refer to memory inwhich no power is required to retain the data stored. Flash memory isincreasingly being used as primary or secondary storage memory incomputing systems. Such devices are commonly known as solid state disks.Flash is also being used as cache memory in some systems.

A solid-state disk drive known to the inventor, but not as publicallyavailable prior art, includes a first portion of solid-state memory of avolatile nature, a second portion of solid-state memory of anon-volatile nature, a controller for managing the memories, and a powersubsystem for protecting data in volatile memory in the event of loss ofpower. The drive is, in one embodiment, a hot swappable disk drive thatis recognized by a host system upon boot as a destination drive forreads and writes.

A controller is provided on the disk drive for managing the memoryportions as a single non-volatile memory through use of at least oneintegrated circuit supporting one or more sets of machine-readableinstructions and a data port and buffer circuitry for bi-directionalcommunication of data between the controller and a host system such as acomputer.

The system known to the inventor uses a RAM/Flash data storageaddressing method that prevents continued and repetitive writing toFlash to preserve mean time before failure (MTBF) of the Flash storagedevice or devices of the system. The system uses RAM as a caching memoryand only writes to Flash when absolutely necessary. Another optimizationof the system is rotation of Flash blocks in and out of service tofurther enhance wear leveling of the Flash device or aggregate ofdevices onboard or plugged into the disk drive.

Writing to Flash memory is comparatively slower than writing to RAM,hence the use of RAM in the above-described system to cache data foreventual write to Flash on power down, power interruption, or only whenthe RAM cache is full. The system described above provides a practicaland economical solution for replacing mechanical hard disk drives incomputers and other devices.

It has occurred to the inventor that there is a need for faster datamanagement speeds in the computing industry in general and in particularin the area of robust servers and other business machines. While CPUspeeds are at all time highs in terms of computing cycles, speeds atwhich data can be managed relative to disk storage on a Flash memory arestill relatively slower. This has caused a barrier to extensive use ofFlash memory in more robust data storage systems.

Still another disadvantage of using Flash memory as long-term storage inrobust systems is that a number of writes performed on the memory islimited on a Flash memory chip and the media must be written in amanner, often proprietary, as specified by the manufacturer of the Flashmemory. Such adaptations may not be readily supported by a particularhost system sending the data for storage. This fact has been a basis fordiffering approaches to Flash memory management software and firmwarethat deal essentially with how data may be rendered and stored on theparticular type Flash memory implemented.

Further to the above, current Flash data storage systems rely on asingle central processing unit (CPU) to manage Flash tables and performother data management tasks. A drawback is that such applications placesignificant performance demands on Flash-based storage and cachingsystems, preventing scaling of those systems to meet enterprisestandards for mass data storage systems.

RAM and specific data bus contentions or issues effectively prohibitsingle processor Flash management schemes from scaling to ahigh-performance level. For example, there are many operations performedby and in conjunction with a microprocessor that compete with each otheron a storage device for RAM space. Error Code Correction (ECC) andreal-time data encryption are just a few of these operations thatcompete with address lookups, read and write requests and otherimportant data access functions.

Current Flash memory research is resulting in faster Flash memory typesthat may be operated much faster than current Flash types. But RAMaccess and data bus contentions present problems in currentarchitectures that cause latency and prevent full potential for fastercomputing. Therefore, what is needed in the art is a Flash-based storagedevice and data management system that can be scaled up forhigh-performance write and read operations without bogging down due toRAM and Bus issues.

SUMMARY OF THE INVENTION

One of several problems stated above is that it in computing where datastorage is involved it is difficult to scale up to larger more robustsystem using a device having a single processor for managing all of theFlash data management operations over a shared bus system. It isdesirable to store data in a fast, secure, and reliable manner usingFlash memory as a preferred solid state medium. However, existingsystems use a single complex processor over a shared bus to access theFlash media for read and write access. The result is a less than desiredperformance speed for read and write operations due to RAM and buscontentions by the various data management process components.

The inventor searched the art of data storage devices and systemslooking for components that could be leveraged or otherwise modified toimprove the flexibility, reliability and performance speed of a datastorage system.

Every data storage drive depends on a processor and a bus system forreading and writing data. Most such systems are not flexible enough tobe scaled up for enterprise applications that might use Flash media asthe persistent storage medium because of bus and RAM contentions.Moreover, larger more complex processors are expensive and althoughcapable of the data management tasks required of enterprise systems,suffer degradation of performance speed at levels of higher utilization.

The inventor conceptualized and subsequently provided a Flash datastorage device constructed using a distributive architecture that wasless expensive to implement and had fewer RAM and bus contention issuesthan single processor devices. The result was a better performancerelative to data management speeds including reads and writes.Accordingly, in one embodiment of the invention, a data storage deviceis provided, comprising a host controller interface, a plurality ofmicroprocessor units each having a portion of random access memory (RAM)dedicated thereto, a plurality of Flash device configurations eachhaving dedicated bus connections to individual ones or multiples of themicroprocessor units, and a dataflow controller accessible to the hostcontroller interface for managing access to the Flash deviceconfigurations.

In another aspect of the invention a Flash channel configured for datastorage is provided, comprising a coupled configuration of one or moreFlash memory devices, a bus structure connected to the Flash deviceconfiguration, the bus including an address line, a data line, and acontrol line, and one or more microprocessor units also connected to thebus structure, and having a portion of RAM dedicated thereto.

In another aspect of the invention a Flash channel configured for datastorage is provided, comprising two or more Flash device configurationscomprising one or more Flash memory devices, two or more bus structuresconnected to each Flash device configuration, the bus structures eachincluding an address line, a data line, and a control line, and a singlemicroprocessor unit also connected to the bus structures, themicroprocessor unit having a portion of RAM dedicated thereto.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

FIG. 1 is a block diagram illustrating a single processor Flash storagedevice according to existing art.

FIG. 2 is a block diagram illustrating a multi-processor Flash storagedevice according to an embodiment of the present invention.

FIG. 3 is a block diagram illustrating a multi-processor Flash storagedevice according to another embodiment of the present invention.

FIG. 4 is a block diagram illustrating a multi-processor Flash storagedevice according to a further embodiment of the present invention.

FIG. 5 is a process flow chart illustrating steps for storing data in amulti-processor Flash device according to an embodiment of the presentinvention.

FIG. 6 is a process flow chart illustrating steps for reading from amulti-processor Flash device according to an embodiment of the presentinvention.

DETAILED DESCRIPTION

The inventor provides a multiple processor flash-based storage deviceand a system for managing data relative to use of the device for datastorage. The invention is enabled in detail according to the followingembodiments.

FIG. 1 is a block diagram illustrating a single-processor Flash storagedevice as known to the inventor. In this system known to the inventorand briefly described above in the background section of thisspecification, a single processor is used to perform all of thefunctions relative to reading and writing data to one or moreFlash-based storage devices.

In the system of FIG. 1 data storage system 100 includes a data storagedevice 101 and a host computing device 102. Host computing device 102may be a personal computer (PC) or a hand-held device such as a personaldigital assistant (PDA), a Laptop computer, or some other computingdevice that can be coupled to data storage device 101 for the purpose ofreading data from and writing data to the device.

Data storage device 101 is a solid state storage device that can behardwired to or can be plugged into the host for use as a disk drive inplace of a mechanical disk drive. Data storage device 101 has a hostinterface controller 103 for adapting to the host system though acomputer bus. Data storage device 101 further includes a microprocessor104 for processing commands from the host. Microprocessor 104 isconnected by internal bust 107 to a random access memory (RAM) 105 usedas a cache memory for the device.

Internal bus 107 connects processor 104 to host interface controller 103and to a plurality of flash-based data storage devices 109 (1-n).Flash-based storage devices 109(1-n) may be flash chips bused in seriesor parallel. RAM 105 is used for all RAM-based functions includingcaching writes to flash for the purpose of lessening the number ofactual writes that the host system makes to flash to preserve thelifespan of the flash storage devices. Data management tables for bothflash space and RAM space are provided in RAM for mitigating writeaddressing and lookups for reading from the flash devices.

In this example writes to flash are kept to a minimum and writing toflash actually occurs in flash dumps from RAM such as when there is apower interruption, a purposeful power-down event, and when RAM space isapproaching capacity. Using RAM as a fast caching system makes theapplication of flash-based storage more practical. However, there arelimitations with this exemplary architecture that prevent this systemfrom economical application to more robust systems like server-basedstorage on an enterprise scale, or mass data storage applications likeredundant array of independent disk (RAID) systems and other like massdata storage systems.

The fact that only one processor is active on data storage device 101coupled with a shared data bus produces certain performance delays indata management relative to processor speed. RAM space 105 is a preciousresource on device 101. Many processes other than data write and readoperations compete for available RAM space. Some of the aforementionedprocesses that contend for available RAM space include data encryption,error correction coding (ECC), and address lookups. Successfulutilization of RAM 105 by microprocessor 104 for all RAM-based dataoperations suffers some degradation as RAM cache fills with pendingFlash writes and as the shared data bus becomes increasingly busy withmore data traffic. Providing more RAM memory is not a viable option inthis example as the shared data bus is only so wide presenting abottleneck to higher performance required for more robust systems.

FIG. 2 is a block diagram illustrating a multi-processor Flash storagedevice 200 according to an embodiment of the present invention. Flashstorage device 200 is a solid-state data storage system using adistributed architecture and dedicated bus structures. Device 200includes a host interface controller 204 in this example that providesan interface to a system host such as a powerful workstation or anenterprise server application. In one embodiment storage device 200 maybe a shared device accessible from more than one computing station orserver. Also in one embodiment device 200 may be part of an aggregationof multiple similar devices to form a server data storage rack or arrayof disks as in a RAID array or in a storage area network (SAN).

Flash storage device 200 may be adapted for use with a small computersystem interface (SCSI) bus, parallel advanced technology attachment(PATA) or serial advanced technology attachment (SATA) protocols,integrated Drive Electronics/Advanced Technology Attachment (IDE/ATA)interface, an Enhanced Small Device Interface (ESDI), a Serial AdvancedTechnology Attachment, (SATA), or a Parallel Advanced TechnologyAttachment (PATA) interface or a Peripheral Component Interface (PCI).Disk 200 may also be adapted to work with enterprise Fibre Channel datastorage networks and serial attached SCSI (SAS) networks. In thisparticular embodiment, disk 200 may be thought of as a solid-state massstorage device using the appropriate form factors and interfaces.

Flash storage device 200 includes a distributed processor architecturecomprising multiple microprocessor units 202 (1-n). Each microprocessorunit 202 (1-n) includes a microprocessor and an onboard or bused accessto a dedicated amount of RAM. The dedicated RAM is used by themicroprocessor in each unit for caching and other data managementfunctions. Microprocessor units 202 (1-n) are intended to be low costdedicated processors that function independently of one another. Eachmicroprocessor has a dedicated bus to one of a plurality of Flashconfigurations 201 (1-n).

The illustration of separate RAM/FMD in each processor unit 202(1-n) isnot meant to indicate that there are completely separate and autonomousRAM units, but simply that each microprocessor unit has a dedicatedportion of RAM. As described above, the dedicated portions might be alla part of a single RAM array. Moreover, Ram portions may be provided onFlash configurations where the configuration is a removable modulecontaining one or more Flash devices and the dedicated RAM. In thatcase, access to RAM would be over a dedicated bus.

A Flash configuration is defined as one or more Flash memory devicesconfigured to be accessible through a dedicated bus. A Flash channel isdefined for the purpose of discussion as a bus connection from aprocessor, for example, to one or more Flash chips or devicesillustrated logically herein, defined as a Flash device configuration orsimply Flash device. Therefore, a plurality of dedicated internal busstructures 205 (1-n) is provided to complete the architecture. Flashconfigurations 201 (1-n) may also be referred to as Flash channelsthroughout this specification.

Microprocessor unit 202(1) is coupled to Flash configuration 1 (Flashdevice) by dedicated Bus 1. Microprocessor unit 202(2) is coupled toFlash configuration 2 by dedicated Bus 2, and so on for the number ofprocessor units (n) on device 200. The ratio of Flash configuration toprocessor is one-to-one over a single bus in this example. However, thisis not a strict requirement for practice of the present invention aswill be detailed further below.

Each microprocessor unit 202 (1-n) has a dedicated bus connection to aunique dataflow controller 203. Dataflow controller 203 manages the datatraffic over all of the Flash channels through each of themicroprocessor units. Each microprocessor unit 202 (1-n) has a baseaddress and is responsible for a single Flash channel of multiplechannels 201 (1-n). The microprocessor units are completely independentand do not communicate with one another in the architecture in thisparticular embodiment. In other embodiments, the microprocessorsdistributed over the architecture may be bused for communication witheach other and may share data and tasks.

Dataflow controller 203 communicates with host interface controller 204by way of a bus illustrated herein as a bus 206. The host system mayview Flash storage device 200 as a single drive or disk or according toany particular partitioning that may be implemented such as primarystorage space and backup storage space. Dataflow controller 203determines which Flash channel to use, that is, which microprocessorunit to use, according to information received in a request andaccording to a Flash management system implemented in RAM in each of themicroprocessor units 202 (1-n).

RAM at each processor unit 202 (1-n) includes Flash management Datatables (FMD) tracking the local block addresses (LBAs) and state for theFlash memory connected to the channel to which the processor unitcontrols access. The actual Flash memory devices may be Phase ChangeMemory or NAND Flash or any other variant of Flash memory or persistentmemory. Such devices may be Flash chips connected in parallel or daisychained, and that are accessible as a configuration through a singlededicated bus. The invention may leverage existing Flash memory typesand newer Flash memory types being developed. The type of RAM used ateach processor may also vary. Available RAM types include SDRAM, MRAM,FRAM, and NRAM. In one embodiment Flash memory may instead be anon-volatile RAM that is suitable for use as a persistent storage space.

Dataflow controller 203 may be a state machine implemented in softwareor firmware. Also, dataflow controller 203 may be implemented asprocessor-controlled hardware. Integration between host interfacecontroller 204 and dataflow controller 203 is also plausible and may bepracticed without departing from the spirit and scope of the presentinvention. Application as a data storage device for a largerenterprise-scale system like a server-based system is among the manyadaptation possibilities for data storage device 200.

The simple one-to-one correlation between microprocessors and Flashchannels in this example is exemplary only as other ratios betweenprocessor and Flash memory may be observed in the architecture. Some ofthese variations are explained more fully later in this specification.

There are several optimization techniques that may implemented relativeto Flash memory management in terms of reads, writes, erasures, and wearleveling. One case for using RAM has a cache memory for parking Flashdata for eventual write to Flash, and uses both RAM address and Flashaddress tables in FMD, as is the case for the co-pending applicationreferenced in the cross-reference section of this application. In oneembodiment dataflow controller 203 selects a processor unit 202 (1-n) insequential order for performing data access. In this scheme a firstrequest will be filled by processor 202 (1), a next request by processor202 (2) and so on. By the time the selection process loops back to thefirst processor, it is most likely free again (free of ongoing dataaccess tasks). A goal is to have maximum throughput of data while notover utilizing or under utilizing any processing resource.

In one embodiment a random selection approach for processors is used. Inthis approach dataflow controller 203 may select a processor forcompleting a write from the host based on a random assignment ofaddresses. In one embodiment wear leveling is practiced in conjunctionwith all of the Flash channels by ensuring that data is evenlydistributed over the collective Flash memory space.

Dataflow controller 203 is asynchronous and may simultaneouslycommunicate with all microprocessor units 202 (1-n). Address and statetables (not illustrated) are provided to the dataflow controller by eachof processor units 202 (1-n). In this way the dataflow controller maymanage where writes occur transparently from the host. The host may viewthe compilation of Flash devices as a single disk according to a filesystem-based view used by the operating system of the host. A moreprimitive view or block view of the Flash memory space may also beordered. It is noted herein that storage device 200 may be one ofmultiple devices comprising a mass storage system accessible from one ormore machines.

FIG. 3 is a block diagram illustrating a distributed multi-processorFlash storage device according to another embodiment of the presentinvention. Flash storage device 300 is illustrated in this embodimentand is implemented using a distributed architecture including multipleprocessor units illustrated herein as microprocessor units 304 (1-n).Microprocessor units 304 (1-n) each have onboard or dedicated RAM forprocessing data management functions and for caching data before writingto Flash operations.

A Flash channel is defined as one or more Flash devices (inconfiguration) connected by a dedicated bus to a processor unit asdescribed above. Each of Flash devices 306 (1-n) represent one or moreFlash memory devices bused to a processor unit by a dedicated bus, inthis example. Flash device 1 and Flash device 2 of devices 306 (1-n) inthis embodiment share microprocessor unit 304 (1). Microprocessor unit304 (1) is bused by a dedicated bus 305 (1) to Flash device(s) 306 (1)to form one complete Flash channel. The same microprocessor unit isbused by a dedicated bus 305 (2) to Flash device(s) 306 (2).

The same configuration is repeated on the device where onemicroprocessor unit is responsible for two Flash channels, for example,microprocessor unit 304 (n) is bused by dedicated bus 305 (m) to Flashdevice(s) 306 (m) and by dedicated bus 305 (n) to Flash device(s) 306(n). In another embodiment one microprocessor unit may handle four oreight Flash channels, or other numbers of Flash channels. There are manypossibilities. In this case RAM is shared for caching writes to bothFlash device configurations (Flash 1, Flash 2). In this embodiment RAMis not dedicated to a single Flash channel but is dedicated to a singlemicroprocessor unit and is shared by two Flash channels. While this mayintroduce some contention for RAM between the Flash channels, the factthat the channel pair earmarked by sharing one microprocessor unit isduplicated over entire device 300 makes any performance degradationnegligible when compared to the performance of a single processor unitmanaging multiple Flash channels over a common bus.

Each microprocessor unit 304 (1-n) has a single bus connection to a dataflow controller 303 integrated with a host interface controller 302. Inthis example dataflow controller 303 is onboard the host interfacecontroller. Dedicated data buses 305 (1-n) may be 32-bit, 64-bit, or128-bit wide buses, or some other bus configuration. The same can besaid for all dedicated internal (onboard) buses described in the variousarchitectures present.

Single bus connection Bus (1) from microprocessor unit 304 (1) todataflow controller 303 may be a 32-bit, 64-bit, or 128-bit wide bus, orsome other. It is possible that Bus (1) may be configured to be twice asfast as buses 305 (1-n) to allow for possible bottle-necking of datatraffic on the host-side of the device 300. Other optimizations may bepracticed such as RAM caching before write where the actual writes toFlash over the dedicated buses 305 (1-n) are kept to a minimum number asmuch as is practical. Bus (1) that connects microprocessor 304 (1) todataflow controller 303 for communication may be a duel independent bus(DIB) or some other bus architecture that is optimized for speed.

Microprocessor unit 304(n) is bused to dataflow controller 303 by adedicated bus n. Dataflow controller 303 includes an onboard processor307 with a dedicated RAM with dataflow controller tables for use inmicroprocessor communication. Dataflow controller 303 is hosted on orintegrated with host interface controller 302. It is not specificallyrequired that dataflow controller 303 be controlled by an onboardprocessor to practice the present invention. The dataflow controller maybe a state machine running in firmware on the host controller interface.The dataflow controller may also be controlled by a processor residingin a host system or in a system adapter without departing from thespirit and scope of the present invention. In this example, eachmultiprocessor unit manages data access to two independent Flash memoryconfigurations. The Flash configuration pairs in this example do nothave to be sequential such as Flash 1 and Flash 2. Microprocessor unit304 (n) controls access to Flash m and Flash n, which may not beco-located on the Flash storage device.

FIG. 4 is a block diagram illustrating a multi-processor Flash storagedevice according to a further embodiment of the present invention. Aflash storage device 400 is illustrated in this example and comprises adistributed microprocessor architecture in which two microprocessorunits are provided to share a single Flash configuration. Each ofmultiple microprocessor units 404 (1-4) in this embodiment havededicated RAM for managing Flash access. In this example there are twoFlash configurations 406 (1-2). Flash configuration or device 406 (1)has a dedicated bus to a bridge controller logically illustrated herein.On the processor side of the bridge, two dedicated bus lines 407 (1-2)connect to microprocessor units 404 (1) and 404 (2) respectively. Inthis example, there are possibilities for managing Flash memory somewhatdifferently than has so far been described.

Microprocessor units 404 (3) and 404 (4) are provided to jointly manageFlash configuration 406 (2). Bus lines 407 (3) and 407 (4) complete theconnection to a bridge that supports the single dedicated bus to Flashconfiguration 406 (2). Moreover, each processor unit pair[microprocessor units 404 (1 and 2); microprocessor unit pair 404 (3 and4)] is bused to a bridge on the side of a dataflow controller 403. Asingle bus from each bridge connects the processor units as selectablepairs to the dataflow controller. The dataflow controller recognizeseach microprocessor unit pair as an entity that controls access to thetotal Flash space of a single Flash device configuration like device 406(1).

In one embodiment the microprocessor unit pair of identical componentsis assigned a specific partition of the total Flash space for eachcomponent. For example, microprocessor unit 404 (1) may be assigned to aFlash partition in Flash configuration 406 (1) reserved for routine datastorage and access. Microprocessor unit 404 (2) may be assigned to therest of the Flash space reserved for data backup and recovery. Inanother embodiment Flash configuration 406 (1) may not be partitioned.Microprocessor unit 404 (1) might be provided to read and write dataonly, and microprocessor unit 404 (2) might be provided to perform otherRAM-based functions like ECC, on device data encryption, updates totables, wear leveling, and other functions. In one embodiment eachmicroprocessor unit in a pair that controls one Flash deviceconfiguration is shadowed with the event activity of the other processorunit so each processor unit knows what the other processor unit isdoing. In this way, work may be divided logically between the twoprocessor units, although they may not directly communicate with oneanother.

In this example dataflow controller 403 includes an onboardmicroprocessor 401 with dedicated RAM and data flow controller tables asneeded for selecting processor unit pairs. The data flow controller isbuilt into or integrated with a host controller interface 402 asdescribed further above with reference to FIG. 3. In this case a singleinternal Bus 1 connects the controller to processor unit pair 404 (1-2),and a single internal Bus 2 connects the controller to processor unitpair 404 (3-4). One with skill in the art of device engineering willappreciate that there may be many more microprocessor units and Flashconfiguration channels included on Flash storage device 400 than areillustrated. There may be more than two microprocessor units dedicatedto a single bused Flash configuration without departing from the spiritand scope of the present invention.

In one embodiment microprocessors are hardwired to provide certainprocessing capabilities relative to Flash management and eachmicroprocessor unit in a pair or grouping assumes a specific processingrole in the management of the Flash device configuration. In anotherembodiment the microprocessor units in a pair may be programmed from aremote interface and set up to perform specific processing roles.Regardless of the underlying assignments, the host system may view thetotal space of all of the Flash configurations as one storage device.Likewise multiple storage devices grouped together as a rack or array ofstorage disks may be viewed by the host as one mass storage device.

Flash management data (FMD) may be loaded into RAM at eachmicroprocessor unit at boot and may include Flash local block addresses(LBAs) and other state data relevant to Flash use. RAM address tablesmay also be loaded into RAM from Flash at each microprocessor in anembodiment where RAM caching is used to minimize writes to Flash duringoperation of the storage device. Dataflow controller 402 may include amicroprocessor with its own RAM for managing data operations at thelevel of the controller such as selecting Flash channels for read andwrite based on requests from the host. All of the appropriate addressand state tables may be uploaded to the dataflow controller from eachactive microprocessor at boot.

FIG. 5 is a process flow chart illustrating steps for storing data in amulti-processor Flash device according to an embodiment of the presentinvention. At step 501 an incoming write request from a host systemarrives at the host interface controller to write data to a storagedevice according to an embodiment of the present invention. At step 502the host interface controller formats the request for the dataflowcontroller responsible for selecting which Flash channel to queue therequest for and hands off the request to the dataflow controller.

At step 503, the dataflow controller performs an internal address lookupto determine where the write will occur. At step 504 the dataflowcontroller accesses the correct microprocessor unit or units and Flashchannel for writing the data. At step 505 the system determines if RAMcaching for Flash writes is turned on. This may be provided as aprogrammable mode on the Flash storage device that may be switched on oroff.

If the system determines that RAM caching is activated at step 505, thenthe microprocessor unit charged with performing the write writes thedata into RAM on the processor at step 507. The RAM on themicroprocessor unit is reserved at least in part for the purpose of RAMcaching if the mode is active. At step 509 the microprocessor unitupdates RAM and Flash tables to record the write and reserve the FlashLBA to receive the write, although no data was actually written toFlash. Steps 501-505, 507, and 509 are repeated for each write requestreceived from the host controller.

The write cycles may continue uninterrupted in this fashion relative toeach Flash channel involving some or all of the onboard microprocessorunits. If there is an interruption of power to the Flash storage deviceat step 510, then all writes to Flash that are valid are written to theappropriate Flash device configurations by the assigned microprocessorunits across the board at step 506. A power interrupt might be anintentional event like system shutdown or re-boot. A power interruptionmay also be an unplanned event such as an inadvertent loss of power. Atime period may be provided or configured for the system to “stay alive”while it is moving data from RAM and writing it to Flash. Some RAM typesretain data without power for a period of time before losing the data.Battery backup may be provided to ensure enough time for the cycle tocomplete.

In one aspect where RAM caching mode is active, one or more of themicroprocessor units on the Flash storage device may inadvertently losepower or fail due to some onboard error or problem. All of the writescached in RAM at the failing microprocessor unit are, in such an event,automatically written to Flash at step 506 for that Flash channel. Oneexception to this rule may be where one Flash channel is a designatedbackup or mirror channel to the failing Flash channel.

If at step 510 there is no power interrupt, then the system may make adetermination at step 511 if there are any full RAM caches on any of themicroprocessor units on the Flash storage device. This determination isongoing as dedicated RAM on one microprocessor unit may enter a fullstate at any time relative to other microprocessor unit RAM caches forother Flash channels. Therefore, this decision may be local to eachmicroprocessor unit and Flash writes may occur at step 506 if the RAMcache is full for that processor unit. As data is overwritten or agedout of validity RAM cache may be purged and periodic Writes to Flash mayoccur on a microprocessor unit to free up the cache.

With RAM caching mode active and the system determination that no powerinterrupt is evident at step 510 and no full cache for a particularmicroprocessor unit selected for write at step 504 is eminent then atstep 512 the system may determine by rule that a particular write isoptimally written to Flash and not cached in RAM at step 512. In thisevent certain data or data types may be marked or flagged by rule foroptimal Flash write overriding RAM caching for that particular writeoperation.

In the event of a write to Flash at step 506, the Flash table is updatedat step 508 along with the appropriate RAM table if applicable. At step512 the process ends for that write request. If none of the conditionsof steps 510-512 are true in RAM caching mode then the process ends forthat write at step 512 and the data stays in RAM until it is purged fromRAM or eventually written to Flash when one or more of the conditionalstates change. If at step 505 RAM caching mode is not active, or no RAMcaching mode is provided, then at step 506 the microprocessor unitwrites the data to Flash. The Flash table for that Flash configurationdevice is updated at step 508 and the process ends for that writes atstep 512.

In one aspect of the present invention RAM caching, if provided, can beselectively turned on or off for each Flash channel configured on theFlash storage device. Optimal writes may be reserved for one or moreFlash channels and RAM caching may be reserved for one or more otherFlash channels. There are many possibilities.

FIG. 6 is a process flow chart illustrating steps 600 for reading from amulti-processor Flash device according to an embodiment of the presentinvention. At step 601 a read request arrives at the host interfacecontroller from the system host. At step 602, the request is formattedfor the dataflow controller and handed off to the controller for furtherprocessing. At step 603 the system performs an address lookup todetermine the correct microprocessor and Flash channel. At step 604, thedataflow controller accesses the correct microprocessor and Flashchannel based on the lookup at step 603.

At step 605, the system determines if a RAM caching mode is active onthe Flash channel. If at step 605 RAM caching is active then at step 607the microprocessor performs an address lookup in RAM by default. At step608 it is determined if the read address is currently valid in RAM. Ifat step 608 it is determined that the data subject to the read requestis stored in RAM cache, then the microprocessor accesses RAM and readsfrom RAM at step 610. The microprocessor returns the read data to thedata flow controller at step 611. In turn, the dataflow controllerreturns the read data to the host at step 612. The data flow controllermay perform one or more operations on the data before handing the datato the host. In one embodiment, the data flow controller may combine theread data with read data from other channels before sending data to thehost depending upon any read optimization scheme that might beimplemented on the data flow controller.

At step 612 the process ends for reading from RAM. With RAM cachingactive at step 608, if the valid data address is not found in RAM thenthe system assumes the data is in Flash. At step 609, the appropriatemicroprocessor unit accesses the Flash channel and the data is read fromFlash. Steps 611, 612, and 613 follow in the same fashion as a read fromRAM.

Referring now back to step 605, if RAM caching is turned off or notprovided, then steps 607, 608, and 610 are not performed. In this caseat step 606 the correct microprocessor performs an address lookup forFlash. The process then proceeds to step 609 where the appropriatemicroprocessor unit reads from Flash. At step 611 the system returns theread data to the data flow controller. At step 612 the data flowcontroller sends the read data to the host. The process ends for thatcycle at step 613. Although reading from Flash does not wear down theFlash memory and can be performed at speeds approaching RAM accessspeeds, reading from RAM with RAM caching turned on may help diminishtraffic on the dedicated buses between the microprocessors and the Flashconfiguration devices. Each Flash channel reports its own state intables to the dataflow controller. Each update performed by amicroprocessor unit is copied to the dataflow controller. Addresslookups can, in one embodiment, be performed at the level of thedataflow controller using internal updated tables freeing up themicroprocessor units for read/write operations and other data managementoperations.

One with skill in the art of data storage devices, particularly thosefor more robust systems, will appreciate the flexibility that thedistributed microprocessor and Flash channel architecture according toembodiments of the present invention provides for a Flash storagedevice. One with skill in the art will also appreciate that multiplelower cost microprocessor units running asynchronously over Flashchannels on a Flash storage device such as those described herein canperform the tasks of a single, more complex processor with morereliability and comparable or superior performance speed due to RAMcaching optimization and dedicated busing to the separate Flash memoriesof the device.

It will further be apparent to one with skill in the art that the Flashstorage and data management system according to various embodiments ofthe present invention may be provided using some or all of the describedfeatures and components without departing from the spirit and scope ofthe present invention. In one embodiment, for example, a systemaccording to the invention could be implemented using the Host CPUs forsome or all of the dataflow controller functions and/or flashmanagement. It will also be apparent to the skilled artisan that theembodiments described above are specific examples of a broader inventionwhich may have greater scope than any of the singular descriptionstaught. There may be many alterations made in the descriptions withoutdeparting from the spirit and scope of the present invention.

1. A data storage device comprising: a host controller interface; aplurality of microprocessor units each having a portion of random accessvolatile memory (RAM) dedicated thereto, the RAM portion having a presetthreshold for valid data; a plurality of non-volatile memory deviceconfigurations each having dedicated bus connections to individual onesor multiples of the microprocessor units; a dataflow controlleraccessible to the host controller interface for managing access to thedevice configurations; wherein the dataflow controller periodicallycompares valid data volume in the RAM portion associated with eachmicroprocessor with the preset threshold, and upon a threshold being metor exceeded, selects and moves data that has been longest in thevolatile memory portion without alteration from the RAM portion to thenon-volatile memory bussed to that microprocessor, and marks locationsfrom which the data is moved as available for writing in the volatileportion.
 2. The data storage device of claim 1 wherein the deviceconfigurations bused to ones or multiples of the microprocessor unitscomprise individual Flash channels selectable for service from thedataflow controller.
 3. The data storage device of claim 2 wherein theFlash device configuration of each selectable Flash channel comprisesmultiple Flash chips connected in parallel.
 4. The data storage deviceof claim 2 wherein the Flash device configuration of each selectableFlash channel comprises multiple Flash chips connected in a daisy chainconfiguration.
 5. The data storage device of claim 1 wherein thedataflow controller includes an onboard microprocessor with a dedicatedamount of RAM.
 6. The data storage device of claim 1 wherein thedataflow controller is a state machine.
 7. The data storage device ofclaim 1 wherein the dataflow controller is hardware with amicroprocessor and a dedicated portion of RAM integrated with the hostcontroller.
 8. The data storage device of claim 1 wherein the memorydevice configurations for each channel are of a type of NAND Flash orPhase Change Memory.